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This paper describes the approaches taken at a middle school 
to improve student problem solving ability and test results for the Indiana 
Statewide Testing for Educational Progress (ISTEP) mathematics tests. The 
focus was on grade 6. In 1996, 35% of the 437 students in grade 6 did not 
master the mathematics problem solving skill. A committee of teachers from 
various disciplines identified what could be done and how to do it to help 
students improve their problem solving and critical thinking skills. Posters 
were designed and posted in classrooms and teachers were encouraged to refer 
students to them as they worked to solve problems. Resource materials for 
teachers were collected and distributed as kits for classroom use. 
Interdisciplinary teams developed problem solving sessions that required 
students to think about real-world problems and demonstrate their ability to 
solve these problems. In the 1996-1997 school year, 69% of students met the 
Mathematics Proficiency standard, and when these students were tested in 
grade 8, 76% met the standard. Results on a local problem solving showed 
continual improvement at each spring and fall assessment. Increases were also 
apparent in student achievement on the ISTEP test with a substantial 
improvement from fall 1997 to 1999. (SLD) 
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Problem Solving Within a Changing Indiana State Testing Program 

By Ronald W. Costello & Daniel W. Chapin 

With 49 states having standards for student performance and 48 of these states 
using standardized tests to measure performance (Achieve 1999), we have to consider how 
we use these standardized test results in the improvement of student performance. 
Furthermore, we have to recognize the need to improve student problem solving skills. In 
our state-testing program, Indiana Statewide Testing for Educational Progress (ISTEP), 
one of the skills tested under mathematics is problem solving which gave us some limited 
data to use. Problem-solving and critical-thinking skills became a target area goal in our 
first round of the North Central Association Outcome Endorsement Cycle at Noblesville 
Middle School (Grades 7 and 8). We had to decide what other measures we were going to 
use and what interventions and strategies to use to improve student problem solving. 

Identifying the Problem 

Indiana students participate in state performance testing at grades 3, 6, 8, and 10. 
There are three parts to the test: 1. ability test (Test of Cognitive Skills); 2. normed 
reference achievement test (Terra Nova); and 3. criterion referenced test (Indiana Essential 
Skills) in Mathematics and English/Language Arts. The Mathematics Essential Skill 
Problem Solving (Standard 9) is defined as: 

Solve problems using an increasing range of strategies, such 

as interpreting a diagram and starting with a simpler problem. 

• Continue to solve problems by strategies, such as making a list, 
drawing a picture, and looking for a pattern. 

• Solve problems that require interpreting a diagram or drawing, using 
logical reasoning, and using guess and check. 

• Solve a simpler problem to suggest a solution to a more complex one. 

• Explore additional problem-solving strategies. 

For the group in this study, we had 35 percent of the 437 students in grade 6 in 1996 not 
master this skill. ( 

Our target area goal was to improve students’ problem-solving and critical-thinking 
skills. Below is the diagram for this target area goal. 
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Triangulation of 
Data 



Data Point Goal 1 




Data Pbint 1: Goal 1 



TriangiiJation iir/oh-ies the use of 
three liita points to support the 
selection of a speciilc '^ai area. 



Data Point 3: Goal 1 



ISTEP Math 



Analysis Based Upon Triangulation of Tlu’ee Data Points 



The first data point was to increase the number of students passing the mathematics section 
of the state proficiency examination which is a criterion reference part of the state test for 
identified essential skills. The second data point was local assessment designed by our 
teachers for problem-solving and critical-thinking activities used in different content areas. 
The third data point was the mathematics achievement score from the standardized testing 
program (CTB McGraw-Hill, Terra Nova). 

Interventions and Strategies 

A committee of teachers from various disciplines was formed to identify how and 
what could be done throughout the school to help students improve their problem-solving 
and critical-thinking skills. Posters were designed and posted in classrooms, and all 
teachers were encouraged to refer students to them on a continuous basis to answer their 
own questions and to solve their own problems as a matter of routine during the school 
day. The committee began to collect resources materials for teachers to use while 
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developing lessons and activities that would require students to understand and use 
problem-solving techniques and strategies. These materials were organized into kits for 
interdisciplinary teams and content areas to use. The committee became the problem- 
solving experts in the school. They previewed teaching materials from all disciplines and 
made recommendations to the faculty on what to use and how to use the materials in class. 

These resource kits developed by the committee became vital as interdisciplinary 
teams and departments established annual goals focused on improving problem-solving and 
critical-thinking skills with their students. Teachers from all disciplines incorporated the 
use of these skills as a normal and regular routine in class. Many of the interdisciplinary 
teams developed team-wide problem-solving sessions that required students to work in 
small groups to think about real-world problems and demonstrate their ability to solve these 
problems. 

Results 

When we analyze student performance results on the three data points, we find 
improvement in all three areas. The results will be reviewed in three categories: 

1. Students meeting the Mathematics Proficiency Standard; 2. Local Problem-Solving 
Assessment; and 3. ISTEP Mathematics Achievement. 

Students Meeting the Mathematics Proficiency Standard 

We had 69 percent of the sixth grade students meet the Mathematics Proficiency 
Standard during the 1996-1997 school year. When they were tested again as eighth 
graders, the percentage of students meeting the standard increased to 76 percent. The 
results are shown in the graph on the next page. We had purchased the data electronically 
from CTB McGraw-Hill so that we could control for students who were not present for 
both testings. Because the criterion-referenced part of the exam is an Indiana examination, 
the software would not allow us to do this. We were able to estimate that this variation was 
probably fewer than ten percent, which was an acceptable level for us. The increase in the 
number of students meeting the standard as sixth graders compared to eighth graders was a 




5 



Page 3 of 7 



z Score of +1.5 which was a substantial or impressive improvement (Data Analysis 
Software, 1999). 



Percent of Students Meeting 
Mathematics Proficiency 
Standard 




a z Score 



This was even more encouraging because the Indiana Department of Education increased 
the items a student had to answer correctly in order to pass the examination in 1999 so we 
had more students in this class who were able to meet the higher state criterion than they 
had as sixth graders. At the state level in 1999, instead of identifying 30 percent of the 
eighth students below the mathematics standard, that number increased to 39 percent. 

Local Problem-Solving Assessment 

As described in the Interventions and Strategies, teachers had planned problem- 
solving and critical-thinking activities in their classrooms. Students participated in a local 
problem solving assessment in the fall and spring of the seventh grade and the eighth 
grade. On the fall testing, students were able to show an improvement of +6.3% on the 
same local problem-solving test. On the spring testing, students were able to show an 
improvement of +8.5% on the same local problem-solving test. We were able to show 
continual improvement in student performance on each assessment, but we need to do more 
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work on aligning the fall and spring test instruments so they are both testing the same skills 
and content. 

ISTEP Mathematics Achievement 

The final data point was to track student progress on the achievement part of the 
ISTEP test. The purpose of this part of the state test was to provide national norms to 
compare student achievement with other students outside of Indiana, and we test locally at 
the seventh grade using the Terra Nova test. For the target group in question, the student 
results using the mean normal curve equivalent score (MNCES) show a substantial 
improvement from the fall of 1997 to 1999. Those results are shown in the table below. 
This increase was a z Score of +1.97 as seventh graders and +2.03 as eighth graders from 
the sixth grade score which was substantial > .30. 



MNCE 

ISTEP Mathematics 




Consideration 

There are a number of limitations and considerations that must be addressed as we 
consider how to address problem solving in the future. 

1 . It is important to be able to use state standardized testing results in being able to show 
student improvement. By all indications, the demands will become greater to show 
student improvement on state mandated tests. 

2. There is a problem with being able to isolate student results over multiple years to track 
progress. We figured the electronic data and software from CTB McGraw-Hill would 
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solve this problem, but it did not. In the future, we must resolve this problem to have 
the most accurate student performance data. 

3. We were to show significant improvement for student performance in meeting the 
Indiana State performance standards and the local assessments. 

4. Although we were able to show improvement in achievement test results, we did have a 
question about the achievement and criterion results. Does the achievement test 
measure the same content as the criterion part of the examination? The state of Indiana 
has made this question less important for us because they have dropped the 
achievement test from the state-testing program in the future so this will not be reported 
at least on state performance results. 



Summary 

We have discovered a number of things about using standardized testing results in 
being able to assess and to show improvement in problem-solving and critical-thinking 
skills. First, by isolating our interventions to an essential skill on the Indiana state 
proficiency exam, we have seen a significant improvement in the percentage of students 
meeting the mathematics standard. Second, we did increase the number of students 
mastering the math proficiency standards as a sixth grader from 35 percent in 1997 to 74 
percent in 1999 which we believe had a direct relationship to the number of students who 
met the math proficiency as eighth graders. Third, we increased the number of students 
mastering Mathematics Achievement test scores as a sixth grader from 56.5 NCMES in 
1997 to 62.6 NCMES in 1999 as eighth graders. 

As the demands become greater on schools and school districts to validate the 
improvement of academic performance using standardized tests, we are going to have to 
deal with and resolve many of the issues, which we have described in this paper. This will 
only happen if educators can find better ways to report and track student performance 
within a single school from one grade level to the next and from one school building to the 
next. We have to be able to demonstrate through student performance data that our schools 
are making a difference in improving performance. 
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